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A PROGRESS EVALUATION OF FOUR BCTV SHOWS 

Bilingual Children's Television (BCTV) Is a federally and privately 
funded project that is In the process of developing a dally television 
series. This series Is being constructed as a supplement to the norrnal 
instructional services provided by schools and Is al^d at early elemen-' 
.ary school children who speak English and/or Spanish. In brief, there 
are five general goals which underlie BCTV's objectives. These are: 

1. To promote cultural pluralism through an understanding and appre- 
ciation of Latin-American culture, heritage, values and mores. 

/ 

2. To provide the Latin-American child with an experience In which 
his or her language and culture predominate, for the purpose of enhancing 
his or her self-concept, encouraging his or her continued or renewed in- 
terest in his or her background, and help to bridge the gap that may exist 
between home and school. , 

3. To provide non-Spanish' speakers with an opportunity to become fam- 
iliar with the Spanish language and to help all viewers recognize the ad- 
vantage of speaking more than one language. 

/ k. To aid the viewer In developing the communication and problem 
solving skills necessary to function successfully In his or her environment. 
' 5. To present selected Information designed to lead to the development 

of concepts In the areas of Human Relations, Food and Nutrition, Natural 

I 

lEnvi ronment, Energy, and Man-made Objects. 

I Each show in the series is composed of a group of program segments that 
f are focused on a central theme, such as the ''need for practice,*' which in 

turn stem from one of the five areas listed In objective number five. A 
' monolingual child (or adult) can readily understand what Is happening in a 
given program segment by watching the action and/or listening to what Is 
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said. Each s^how lasts for 30 minutes and Involves animation, film excerpts, 

4 

and "live action" in its several segments. 

In an evaluation of the two prototype (pilot) shows for the BCTV series, 
it was ascertained that the shows' basic format was generally entertaining 
and, attractive; and, the content was understood by the children in the tar- 
get audience (BCTV, 1973). The evaluation of the prototype shows did not 
examine whether they were achieving their Instructional objectives since it 
was anticipated that significant learning effects and attitude changes would 
not become evident until a child had had an opportunity to view a substantial 
portion of the whole series. However, BCTV desired some preliminary Infor- 
mation with respect to the attainment of objectives as soon as several shows 
were esseatially ready for distribution. This need arose from the desire to 
have a sound basis for making decisions regarding possible modifications In 
the general developmental strategy. Thus, the specific purpose for the pre- 
sent evaluative-research was to assess the extent ^o which four of the ini- 
tially developed shows were making progress In achieving BCTV's objectives. 

Procedures 

The basic design for this study .Involved the use of Individually admini- 
stered criterion .referenced pretests and posttests to semi-Intact groups of 
control and experimental children In the target populations. The experimen- 
tal group saw each of the four shows twice while the control group did not 
see any of the shows. 

Subjects 

Six year old children In the first grade who spoke English and/or Span- 
slh at each of four sites were used In this study. The four sites (Albuquer- 
que, New York City, Salt Lake City, and San Jose) were selected to provide 



geographic and cultural variety. The children at each site were assigned 
to experimental and coritrol conditions In randomized clusters In order to 
minimize assignment bias and yet not Interfere unduly with the schools^ 
routines. a 

At the beginning of the pretest, the language dominance of each child 
was determined (i.e., English, Spanish, or Bilingual). The standardized 
procedures used for this purpose are presented In Appendix A. The evalua* 
tlon plan called for 10 students In each of the deslgn^s ^8 cells (i.e., 
2 groups X sites X 3 categories of language dominance X 2 sexes). The 
actual number of subjects In each cell who completed both the pretest and 
posttest are presented in Table 1. The n*s for children In the experimen- 
tal group In this Table refer to Just those children who saw all four 
shows twice. ^ 

Evaluation Test Construction 

In order for scores on the evaluation tests to be veridical reflections 
of the effects^^ the shows, each Item of the test was criterion-referenced 
to a particular goal of the shows. Representatives of Bilingual Children's 
Television, Inc., Educational Evaluation Associates, and the Institute of 
Cultural Pluralism viewed the shows and then consensually constructed test 
Items that could reasonably be assumed to be sensitive to the effects of 
the shows. Specific concerns underlying the Items so developed were: 

1. Age Appropriateness . The Items should be appropriate, both In con- 
tent and difficulty, for children at age six. This meant that the conser- 
vation Items had to be sufficiently difficult so that not all children would 
get the I terns correct, regardless of the shows. In addition, the language, 
both English and Spanish, had to be carefully selected to be well within 
the receptive repertoire of six-year old children. 



2. I ns t ruct i ona 1 I ndependence , Although long term conceptual develop- 
ment is the ultimate goal of most educational programs, including the BCTV 
series, the time constraints on the evaluation called for the assessment of 
immediate learning of the shows* content and intent. The general izabi 1 i ty 
of the learninQ was to be assured by the construction of items with the 
same content as the shows, but with different formats and applications, in 
this way, scores on the Items would not represent merely a short-term rote 
memory effect. Thus, item presentations were constructed so as to be sig- 
nificantly different from presentations used In the actual shows. By this 
means, It could be assurred that the child who correctly answered a test 
item w|s in fact transferring what he/she had learned from the show to a 
novel setting which dlfferred In method of presentation while still main- 
taining the essential educational intent of the show* 

3. Bias . To avoid cultural and sex biases In the items, names of the 
characters were selected to be common to both English and Spanish-speaking 
cultures; and, the characters were balanced as to sex membership. The test 
development team also represented diversity tn language dominance, sex, and 
cultural backgrounds. 

Objectives Measured . The funding and time constraints on this study 
did not ^fl low for a complete assessment of all of the BCTV goals. Thus, It 
was decided to sample a range of objectives from the most general to the 
most specific in three general areas of concern, namely: Labelling, Con- 
cepts, and Attitudes. Within these areas, the specific objectives chosen 
were the ones that were emphasized the most across all four shows used In 
this study. 

5. Practical Ity . There were a number of unique constraints that had to 
be taken Into consideration In the design of administration of the tests. 
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such as the number of shows available for distribution St the Inception of 
the study, the availability of classroom time for viewing the shows towards 
the, end of the school year, the availability of a cadre of bilingual exami- 
ners at a given site as well as classrooms containing adequate numbers of 
Spahlsh speaking or bilingual children (so that normal school operations 
would not be disrupted), and related concerns. J 

« 

The Evaluation Test. 

The evaluation test was constructed In parallel English anid.,Spanish 
forms. This was done so that the child took the test In the language In 
which he/she was dominant. In the case of a bilingual child, that child 
was given the test form corresponding to the language In which he/she had 
expressed a preference to be tested. 

The format for most of the Items Involved the examiner's reading of 
the Item in the test booklet while the child followed Illustrations In a 
separate booklet common to both the English and Spanish forms. The child 
then, indicated his/her response verbally or by pointing to a picture or 
object in the separate booklet. The attitude Items were an exception to 
this procedure and are discussed In more detail below. 

In order to stay within the desired 10 to 20 minute administration time 
period, the test was limited to 12 items. Some of these Items, however, 
yielded several points of credit. Copies of the complete set of test and 
scoring materials for the English and Spanish Forms are contained In Ap- 
pendices B ind C, respectively. Appendix A contains the booklet common to 
both forms . 

The three major areas for which test Items were constructed and the 
specific objectives within each area assessed were as follows: "Labelling" 
(Number Identification), "Concepts" (Number Conservation and Content Theme), 
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and "Attitudes'* (Towards English and Spanish languages). A description of 
the five subtests used to measure these objectives Is presented below: 

K Number Identification (Score range: 0 to 8 points). Periodically 
within each show, a segment was concerned wi-th__learni ng the name of a numer- 
al was also embedded into aspects of short story sequences. The three Items 
used to measure this area (Items C, E, ,and J) Involved naming the numeral 
1 1 lust rated, in both languages and Identifying the numeral spoken in the non- 
dominant language. The numerals used for these Items were those that were 
emphasized in the shows. 

2. Number Conservation (Score range: 0 to 2 points). The numeral em- 
phasized within a given show also was visually permuted In many ways to il- 
lustrate, in an indirect manner, that It conserved Its number quality through- 
out the permutations. The two Items used to measure this area (A and G) in- 
volved the child recognizing that the, number of objects In a situation re- 
mained the same even though they had been moved several times. The numerals / 
used for these items were among those that were emphasized In the four shows 
employed In this study. 

3. Thematic Content (Score range:. 0 to 20 points). The basic under- 
lying theme of each show focused upon a specific topic. The four topics 
represented by the shows used in this study were concerned with energy and 
work, pollution and ecology, the need for practice, and the varied uses of 
cloth. The five Items used to measure this area (B, D, F, H, and I) Involved 
a variety of tasks, such as Identifying skills that did or did not need prac- 
tices to develop and identifying pollutants In a nature scene. 

Ii. Positive Bicultural Attitudes (Score range: 0 to 3 points -per lan- 
guage). The procedure employed to assess attitude change was an adaptation 
of the method used by Lambert (Lambert, 1967; Lambert, Frankel , & Tucker, 



1966). This technique is a variation of the semantic differential approach 
to attitude assessment. It Involves the child hearing a short discourse in 
onTTahguage and then rating the speaker on a set of person^ characteristics 
The child then hears the same speal<er with the same discourse, but in the 
other language. Since speaker and content are the same, differences In the 
tivo sets of ratings of personal characteristics are presumed to be attribu- 
table to attitudes held by the child toward the languages and their concom- 
ml tant cul tures . 

In the present study, an uneventful and common passage was selected for 
the discourse. Two audio tapes were constructed. The first tape contained 
a female speaker saying the passage in Spanish and then In English.-: The 
second tape contained a male speaker saying the passage In English and then 
in Spanish. Female examinees heard the female speaker and male examinees 
heard the male speaker. Upon hearing the speaker, the child pointed to 
either of two words on each of the following four dimensions: short-tall 
(practice Item, not scored); good-bad, dumb-jmart, good looklng-uy. /. 

Test and Show Administration 

With few exceptions, the evaluation schedule was. pretest on a Friday, 
presentation of the shows to the experimental group on the following Monday 
thrp<9S Thursday, and then posttest on Friday. Each show was shown twice to 
ti^e experimental group In a counterbalanced manner so that a given show was 
seen both in the morning and the afternoon and there was at least a one full 
day sepa¥ation between each. viewing of a given show. The shows were presen- 
ted by means of closed circuit or (portable video equipment. 

The testing was done on an Individual basis with examiners who were flu- 
ent In both English and Spanish. Each examiner used a portable cassette re- 
corder to play the evaluation test tapes of the male and female speakers to 
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male and female children, respectively. In general, the examiners did not 
know whether a given child was in the control or experimental group; and, 
each examiner tested children In both groups. 

A fully bilingual coordinator was assigned to each site. The coordina- 
tors' responsibilities included establishing liaisons between the schools 
and the independent evaluatl^Q^a'gency , presenting the shows, training and . 
supervising the examiners, keeping a log of student reactions to the shows 
and any anomalies, in the testing procedures, and related functions. A sum- 
mary of the coordinators' logs are presented In Appendijf-J. 



Analysis Procedures 

The data analysis plan for this study^called for an analysis of covari- 
ance (ANOCOV) for each of the five dependent >^arlables; I.e., the subtest 
scores for number identification, number conservation, thematic content, ^ 
positive attitude toward English Language, \and posltlv^ attitude toward Span- 
ish Language. Each of the ANOCOV's was to ^ave Involved the fol lowl ng four 
factors: Treatment Group (control vs. exper^l mental ) , Language Oomlnpnce 
(English, Spanish, and Bilingual), Sex, and GeographU Site (Al buquejrque. 
New York City, Salt Lake City, and, San Jose). The corresponding pretest 
score for each posttest dependent variable was to be used as the covariate 
to adjust for any possible differences between children In their ability to 
answer the evaluation test ^questions . 

As may be seen from an Inspection of Table 1, there were marked differ- 
ences in the obtained cell sizes even though the total number of children on 
which there was complete data {H'kjB) was essentially the same as planned. 
The discrepancies between planned and actual sizes were largely due to dif- 
ferences among the number of children In each of the three categories of 
language dominance. This marked disproportional i ty came about as a result of 



numerous systematic and extraneous factors, such as having to drop 25 bi- 
lingual children at Salt Lake since they took the pretest in Spanish and 
the posttest in English. Similarly, about 30 students were lost at the San 
Jose site due to an epidemic of mumps! Thus, i^ was no longer feasible or. 
appropriate to analyze the complete factorial design for each dependent var- 
iable. In the place of the planned set of analyses of the_^,ompIete design, 
a series of ANOCOV's with unequal cell sizes were employed to test the four 
main effects and the two factor' interactions Involving the treatment group 
factor. All other interactions were pooled Into the error term. This ap- 
proach allowed maximum a/ialytlcal precision with respect to the primary Is- 
sues, I.e., did the Experimental group do better than the controls; and If 
they did, was this trend consistent across sites, sexes, and categories of 
language dominance? ^ 

\ 

Results j 
A summary of the results of the five analyses of covariance are presented 
in Table 2. These results Indicated a significant difference between the 
treatnient groups ir. two of the three areas, of concern, namely: Labelling 
and Concepts. An inspection of the adjusted posttest means scores presented 
In Table 3 indicates that the experlmei>tal group perforined better than the 
control group In both instances. Thus, the TV shows did have the desired 
effect in these two are^. 

An examination of Tab^e 2 further reveals that there were no slgni^Jcant 

\ 

interactions between the treatment group factor and the other\three factors. 

\ 

In other words, the effect of the treatment was consistent acrciss sites, sexes, 
and language dominance groups. 

The results presented in Table 2 also Indicate that there were systematic 
differences between sites and between language groups In how much they Improved 
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in performance from pretesting to posttesting (I.e., irrespective of whether 
or not they were in the experimental or control group). An Inspection of the 
cell means revealed that the differences between language groups was largely 
due to the English dominant children gaining more than the others. The dif- 
ferences between sites also >appea red to be confounded with this trend in 
that the sites that showed the largest gains were also the ones with propor- 
tionately more English dominant children. 

In two instances, there was a significant difference between males and 
females, namely, girls improved more than boys In number Identification and 
in developing positive attitudes towards Spanish. However, none of these 
gains was in any way related to the treatment; and, therefore, of little in- 
terest to the major purpose of this study. 

Discussion 

The foregoing results indicated that the presentation of the shows did 
have the desired impact in the areas of labelling and concepts, but not in 
the area of the development of positive attitudes. The assessment of label- 
ling skills was limited to number Identification, I.e., could a child learn 
the word for a number in another language? Although a given number was em- 
phasized in each of the four shows. It was not the same number. Thus, even 
though a child in the experimental group saw each show twice, the degree of 
exposure to each number was still relatively brief (particularly with respect 
to the memory span of a six year old child). Despite this handicap, the ex- 
perimental group did perform better than the controls; although an Inspec- 
tion of Table 3 Indicates that the absolute size of this advantage Is not 
especially Impressive. ^ 

Within the area of Concepts, the experimental group did better than the 
controls on one of the two dependent variables, namely, content theme. As 



in lh<; case; of number identification, the absolute size of this advantage 
Uboul one point out of 20) was statistically significant but not particu- 
larly large from an educational point ^f view. There are several reasons 
which may account for, this kind of finding. For example, as noted in the 
Introduction, learning effects may only become truly evident after the 
children have seen a substantial portion of the total set of 65 shows. The 
reason for this is that such exposure would provide adequate reinforcement 
since there is similar content across the full set of shows. Further, the 
shows are not didactic. They endeavor to get the labels, Ideas, and atti- 
tudes communicated in an indirect and a child-appealing manner so that the 
shows are not perceived as just another extension of "school." An addltion- 
al limiting factor is that it Is generally difficult to get reliable test 
data from six year old children even when they are tested In the language 
•In which they are dominant. The fact that 25 students at one site "switched" 
their language dominance in one week is Illustrative of this problem. This 
kind of instability (unreliability) tends to mitigate against finding sig- 
nificant differences since it diminishes the sensitivity of the test ques- 
tions to uncover real differences between children. Thus, given all these 
limiting factors, the finding of even small consistent differences between 
the experimental and control groups Is Indeed noteworthy. 

The failure to find significant differences across treatment groups In 
the measure of number conservation also may be attributable to some of the 
limiting variables noted above as well as to certain unique factors. One/ 
of these factors was that there were relatively few points assigned to meas- 
uring this concept and this In turn limited the degree to which differe<jces 
might be found (i.e., via the limitation of the available systematic vai-i- 
ance) . Siegel (1969), Piaget, and others who have experienced difflcujty In 
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educating children with respect to **conservat ion** also have come to the con- 
clusion that this concept may not be directly tralnible; thus, it would al- 
most be presumptive to expect that the indirect methods employed in the BCTV 
shows to achieve this end would be successful* If this is indeed the case, 
then BCTV might well reconsider continued inclusion of number conservation 
segments in its shows. On the other hand, the segments within a show deal- 
ing with number conservation also focus on the same number that Is emphasized 
by that show for the purposes of number identification; and, it may be argued 
that the number conservation segments provide valuable reinforcement for the 
learning of the labels* Subsequent research on this Issue Is needed, es- 
pecially given the amount of show time devoted to number conservation segments. 

The lack of significant differences In the attitude area may be attribu- 
table to many of the same factors as noted above, such as score unreliability 
and limited variance, as well as to certain special problems associated with 
attitude measurement. For exampU, most of the children **topped out" on both 
the English and Spanish scales in the sense that they were quite positive 
about both "speakers." This finding may be more an artifact of the assess- 
ment technique with young children than any true Indication of their atti- 
tudes towards persons who speak these languages; or, It may be that such 
children have not even developed attitudes that discriminate between English 
and Spanish. This is an important consideration for BCTV since It is one of 
the primary goals of BCTV to develop such attitudes. It was also recognized 
at the inception of this study that finding differences In this area after* 
only two showings of four shows was quite unlikely; however, given the cen- 
tral importance of this area. It was deemed necessary to make an attempt to 
find possible impacts. This point of view also led to the requirement for 
each site coordinator to keep a log of what occurred during both the shows 
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and the test sessions. A summary of these logf U presented in Appendix 0 
and it indicates that in general the children did enjoy watching the shows 
and often actively engaged in activities associated with them, such as sing- 
ing the songs. 

Conclusions 

The results of this study Indicated that the four shows were effective 
in improving student performance In the areas of number identification and 
content theme. No significant differences were found between the two groups 
in the areas of number conservation and the development of positive attitudes 
towards English and Spanish. The failure to find a positive effect on the 
attitude measures appeared to be due at least partly to the ctirtailed range 
of scores on \hese measures. Thus, no conclusions may be drawn about the 
possible effecXveness of the shows with respect to their Impact on student 
attitudes. An examination of the coordinator's logs does reveal, however, 
that the students usually enjoyed the shows and It Is presumed that such In- 
volvement may eventually have carry-over effects In the sense of Improving 
student attitudes. Finally, it shouW be noted that .he present study only 
involved the presentation of four dlff6rent shows (even though each one was 
shows twice). This point is Important since the philosophy underlying the 
BCTV series Is that the major beneficial effects of the shows will only be- 
come fully evident after' the child has seen a substantial portion of them. 

In conclusion, it appears that the shows are generally making satisfac- 
tory progress in achieving at least some of their general goals. 
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